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ABSTRACT 

An 18-month project was conducted to create a 
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Vocational Education's Career Planning System (CPS) designed for use 
with mildly mentally retarded, learning disabled, and behaviorally 
handicapped middle school students. Separate formative and summative 
evaluations measured the effectiveness of the microcomputer version 
of CPS. During the Normative evaluation, researchers administered 
exit interviews to 49 students completing the 10-week pilot test of 
CPS and asked the 7 teachers at the 5 pilot test sites to complete 

were gathered through two evaluation 
impact of the CPS materials were 
of pre- and posttests to students in 
J and data concerning the teachers* 
assessments of the programs' overall quality were elicited through a 
five-point rating scale that was included on the instructor log. 
Students and teachers alike found the microcomputer version of CPS 
interesting, enjoyable, and usable. The pre- and posttest scores of 
the experiment group further attested to the effectiveness of CPS. 
(Copies of the e/aluation instruments used are appended.) (MN) 
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FOREWORD 



Career planning for handicapped people has become increas- 
ingly important over the la^it decade because legislation, liti- 
gation, and a growing public^ awareness have all combined to 
promote the full participation of handicapped citizens in the 
mainstream of American life. This project, which has resulted 
in the production of a comprehensive microcomputer-based career 
planning system for mildly mentally retarded, learning dis- 
abled, and severely behavioral ly handicapped middle-school 
students,, has thus provided an important and timely tool to 
assist these students as they begin to explore careers and to 
make educational decisions that will have significant impact on 
their adult lives. Moreover, by providing for the delivery of 
career planning experiences through the latest in microcomputer 
technology, the project has ensured that its target students will 
also reap the wide range of learning benefits this technology 
offers. 

Appreciation is expressed to all project staff members who 
have participated in the eighteen-month effort to design and 
produce these materials: to Dr. James P. Long and Dr. Leonard 
0. Nasman, who directed the project; to Delia Neuman, Jon 
Persavich, and Wheeler Richards, graduate research associates who 
participated in the project's instructional-design phase; to 
Patrick Hurley, Brent Miller, Devin Needles, Don Kreinbrink, and 
Dave Keen, who programmed the student materials; to Gail 
Merecickey, who did the project's graphic design; and to Margaret 
Barbee, Carolyn Goodrich, and Cathy Martin, who provided secretarial 
support. 

Appreciation is also expressed to the' -teachers and other 
staff of the five school districts that participated in the 
project's field test; their cooperation Was essential to ensuring 
the quality and validity of the final instructional materials. 



Robert E. Taylor- 
Executive Director 
The National Center for 
Research in Vocational Education 
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EXECUTIVE SUMMARY 

The field test of the Ca^;|^r Planning System (CPS) , Micro- 
computer Version, involved the ''collection and analysis of data 
reflecting the effectiveness of the materials for two purposes: 
(1) formative evaluation and (2) summativa evaluation. This 
report first describes all the instructional materials that 
comprise the CPS, Microcomputer Version* Next it describes the 
project effort involved in accumulating and utilizing both 
foriTiative and summative data: Finally, it analyzes both types 
of data, providing conclusions and recommendations. 

The description of the instructional materials includes an • 
explanatior). of the software. St udent Guide , and Instructor Guide . 
Tables and' Figures are also provided to enable the reader to 
comprehend how student, teacher, and microcomputer all interact 
in an effort to promote career awareness. 

The description of the formative evaluation effort explains 
how an adaptation plan guided the original development of the 
system, whereas the pilot-testing process provided valuable 
guidance for its subsequent revision. The summative evaluation 
effort is described' next: formation of control groups and 
experimental groups at each test site; experimental design; the 
use of pretests and posttests for students; testing of the 
hypothesis by means of a standard statistical analysis; and the 
provision for ratings by teachers. 

The analysis of the formative data consititutes a major 
portion of this report. The data supplied by the Instructor Log 
from each pilot-test site was studied and categorized for every 
segment of the system. The suggestions for improvement and 
criticisms made by the instructors, as well as those made by the 
students and transmitted through the instructors, are all re- 
corded for the purpose of suggesting revisions in the software 
and print materials. This section records how many students 
actually completed certain portions of the system. The results 
of this data provided the basis for extensive revision of the 
materials during the last few weeks of the project. 

The analysis of the summative data provided a basis for 
accepting the hypothesis upon which the project was based. It 
had been hypothesized that interaction with the CPS, Microcomputer 
Version, would significantly advance target student knowledge 
about careers, about self with respect. to careers, and about the 
career planning process. Student growth, measured as the dif- 
ference in mean scores from pretest to posttest, was compared for 
the control group and the experimental group. The difference was 
found to be statistically significant at the .05 level. This 
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allows the acceptance of the hypothesis when data from all five 
pilot -test sites are aggregated. No other statistically " signifi- 
cant results were discovered. 

Ratings made by instructors are also compiled and summarized. 
The CPS^ Microcomputer Version, was rated favorably by all 
instructors . 

Five conclusions and eight recommendations were developed as 
a result of the project. Perhaps the most striking conclusion is 
that the system is usable for a wider audience than the group with 
which it was pilot-tested. It is recommended bhat .1 subsequent 
project undertake the process of developing an alternative 
management system as suggested by some students and teachers. A 
second significant recommendation is that print-based materials 
should not be adapted to the microcomputer, since the inevitable 
tendency is to make the microcomputer an electronic page turner. 
Other reconunendations are made 'to assist researchers and developers 
who undertake future projects of a similar nature. 

The report includes references, and it concludes with an 
appendix that provides copies of the evaluation instruments used. 
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TOTRODIJCTION 

During the fall term bf the 19 82-1983 academic year, five 
school districts in or neap, major metropolitan areas participated 
.in the field test of the Career Planning System (CPS) , Micro- 
computer Version — a comprehensive, individualized career guidance 
package for mildly mentally retarded, learning disabled, and 
behaviorally handicapped middle-school students. The package was 
adapted to the ^ Riicrocomputer from a print-based product previously 
found successful with such students. For the field test, seven 
teachers and a treatment group of forty-nine students in the 
participating districts used the CPS materials and provided a 
variety of data about them. A total of twenty-three students from 
the same districts served as the control group against which the 
progress of the students in the treatment group could be measured. 

The participating districts were as follows: 

• Boulder Valley Public Schools 

Boulder (Denver) , Colorado ^ 

• Brentwood Public Schools 
Brentwood (St. Louis) , Missouri 

• Columbus Public Schools 
Columbus, Ohio 

(i^^ m Farmers Branch — Carrollton I . S. D» 
Carrollton (Dallas) , Texas 

• Freeport Public Schools 
Freeport (New York City), New York 

The purpose of this evaluat*^n report is to present a 

summary and analysis of the information collected during the 
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field test in relation to (1) the materials' effects upon the 
goal achievement of the treatment group and (2) the perceptions 
of these students and their teachers regarding the strengths and 
weaknesses of the materials. This report will have direct value 
to the U.S. Department of Education, the sponsor of the project 
under whose auspices the materials were developed and tested.. 
The report should also be of interest to other researchers in- 
volved in similar research and to the five districts that partici- 
pated in the field test effort. The report has also provided 
National Center staff members with data to guide their efforts in 
revising the materials to make them more useful for teachers, of 
the learning-handicapped as well as more valuable for students. 
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DESCRIPTION OF THE INSTRUCTIONAL MATERIALS 



The Career Planning System (CPS) , Microcomputer Version, is 
a comprehensive, individualized career guidance package for 
middle-school students who have been identified as mildly mentally 
re^iarded, learning disabled, and/or behaviorally handicapped and 
who road at approximately the 3.5 grade level. This version of 
CPS is designed to involve these students actively in planning 
their own career-related studies and other activities for the 
future. The CPS revolves around the following four major career 
development outcomes: 

• To learn about personal interests 

• To examine occupations to learn how they 
may relate to personal interests 

• To identify interests and areas of study 

in which personal interests may be nurtured 
and in which related skills may be developed 

• To relate educational preparation to possible 
occupational choices 

The CPS, Microcomputer Version, package consists of three 
types of materials designed to foster the attainment of these 
outcomes: (1) a set of thirty computer floppy disks that contain 
the major management and instructional components of the System, 
(2) a Student Guide for each student wh ases the materials, and 
(3') an Instructor Guide for the teacher or counselor who guides 
and facilitates the students • exploratory and planning activities. 

The floppy disks---which were devel-^ped fdr use on the Atari 
800 nicrocomputer system--console with 64K RAM of memory, 
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television sot, and two disk drives — include three management and 
cwenty-seven instructiona]. disks collected •. loose-leaf pocketc 
and enclosed in a bin(^er. The management component is a compre- 
hensive, easily used system that allows the teacher to create a 
class roster, keep track of student data and progress throughout 
use of th^ CPS, alter some aspects of the CPS as necessary to 
serve student and class needs, and so forth. The complete menu 
of management functions included in this component is listed in 
Figure 1. 

Once students sign on to the computer to use the CPS, they • 
are routed automatically through the twenty-seven disks that 
comprise tl^e System^ s instructional activities: Introduction, 
Interest Sort, Interest Areas, and Education Plan. Figure 2 dis- 
plays this general sequence of student moVement and lists the ten 
Interest Areas the students can explore through the CPS. 

The Introduction material consists of activities designed 
primarily to familiarize the student with the CPS — its goals, 
structure, procedures, and vocabulary. This section includes 
such activities as chartLi, a word-search puzzle, and cross rd 
^puzzles that require the student to participate actively in the 
acquisition of concepts and vocabulary essential to using the CPS 

The Interest Sort presents a series of fifty statements 
about aspects of various CPS occupations (working outdoors, 
helping people with their problems, and the like). The students 
respond to these statements by indicating their level of interest 

« 
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Ficjure 1. Instructor Options 
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INTRODUCTION 
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Figure 2. Career Planning System Overview 
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in each — like "a lot," or "a little," or "not at all," After 
the students have entered all these responses^ the computer com- 
piles the Into a ranked listing of as many as ten Interest Areas 
for each students. The students' computer-directed movement 
through these areas in the order indicated by their own responses 
constitutes the core of the CPS individualized career-^planning 
experience. 

The Career Planning System 
The Interest Areas themselves form the main body (twenty-six 
disks) of the Career Planning System^ as well as its conceptual 
core. These Interest Areas highlight interests that middle- 
school students may have or may wish to explore, rather than 
focus on traditional occupational clustering systems. The ten 
CPS Interest Areas are designed to help students acquire immediate 
self-knowledge rather than to speculate about the future. The 
occupations included within the Interest Areas were also chosen 
to represent rather than exhaust the world of work. They cover 
a wide range of job tasks requiring visual, verbal, and computa- 
tional skills, and encompass an educational continuum from high 
school diploma to advanced degree, as well as a range of skills 
from entry*-level to professional. Figure 3 indicates the scope 
and organization of this central component of the Career Planning 
System^ Microcomputer Version. 
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1. Advising 



6. Maintaining and Repairing 



• Child care attendant 

• Employment counselor 

• Lawyer 

• Travel agent 

* 

2. Arranging 

• Architect 

• Florist 

• Hair stylist 

• Interior designer 



• Appliance repairer 

• Mechanic 

• Plumber 

• Sanitation worker 



7. Thinking in Pictures 

• Cartoonist 

• Cominercia] artist 

• Display artist 

• Photographer 



3, Building and Maldng 



8. Using Envlronmentai information 



Carpenter 
Drafter 
Painter 
Roofer 



Farmer 

Fish and game warden 
Landscape gardener 
Meteorologist 



4. Doing Clerical Work 

• Cashier 

• File clerk 

• Secretary 

• Stock clerk 



5. Helping 

• Licensed practical 

nurse 

• Police officer 

• Properties manager 

• Waiter 



9. Working with Equipment 

• Computer service 

technician 

• Cook 

• Machinist 

• Telephone operator 



10. Working with Numbers and Symbols 

• Library assistant 

• Market research coder 
^ Medical laboratory 

assistant 

• Teller 



Figure 3. CPS Interest Areas and Occupations 
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[•'iqiirc 4 doscrlbos the fuxjuonco of student, inovoinent. thro\Kjh 
each Interest Area. First, the students encounter a Probe — a 
brief introduction to Various aspects of the Interest Area — and 
meet four representative workers from the Area whose jobs the 
students can elect to explore. The students can then choose to 
complete one of four interactive Activities designed to simulate 
the job tasks of these workers and to give the students a "feel" 
for the actual procedures of the jobs. If this Activity excites 
the students' interest in the occupation, they can choose to 
proceed to a menu-driven Brief, through which they "interview" 
the worker by asking several specific questions about informa- 
tion that is important to consider when making career choices 
(e.g., additional job tasks, work environment, educational 
preparation., salary range, and so forth). 

After exploring as many Activities and Briefs in the Interest 
Area as the student? desire — and, in the process, completing a 
simple Reaction Form describing their feelings about each — the 
students talk briefly with the instructor before moving on through 
the automatic routing supplied by the CPS management system to the 
next Interest Area on their individualized lists. This talk is 
termed an "Exit Interview"; one occurs at each conclusion of a 
student's exploration of an interest area. 

When the students have explored all the Activities and Briefs 
of interest in their identified interest Areas, the final CPS 
component (the Education Plan) is encountered. First, the 
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Figure 4. IntGrosL Area Flowchart 
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students are romindeti that the coinputor wilJ use their rosponsos 
on the Reaction Forms to compile a list of all the jobs (up to 
a maximum of ten) in v^hich each student indicated a high level 
of interest. The students copy the^e lists into their Student 
Guide and take this i .formation to the teacher or counselor. The 
teacher or counselor works with them to synthesize the information 
gained through. the exploratory experience, to plan some additional 
activities, and to select some basic courses that will help the 
students examine possible occupations in greater depth. ^ 

The Student Guide 
The Student Guide , which provides a permanent recotd of the 
students' final lists of occupations of interest, serves a 
variety of purposes earlier in the students' use of the CPS, as 
well. The only hard copy the students retain throughout the 
program, the Guide (1) introduces them to the CPS and to the 
procedures for loading a disk; (2) provides work she-ets for 
practice and reinforcement for the concepts and vocabulary 
presented iix the computerized Introduction; (3) includes a glos- 
sary of words (e.g., wage, salary, college, apprenticeship) that 
students encounter in the CPS Briefs; (4) provides a central 
record of the students' CPS experiences, reactions, and developing 
interests; (5) offers an opportunity for the students to expand 
upon the reactions entered in summary form through the comput- 
erized Reaction Forms; and (6) includes a section into which the 
students can copy uhe courses and a^jtivities to pursue in order 
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to enter occupations of interest. To conclude their CPS experi- 
ences , the students take the Guide to the teacher or counselor 
and work together to use the recorded information, fillinq out 
the final section of the Guide — the Education Plan. 

The Instructor Guide 
The third kind of material included in the CPS package is 
the Instructor Guide , which provides the theoretical, technical, 
and instructional information a teacher or counselor needs to 
implement the CPS. The various sections of this Guide provide a 
basic awareness of the goals, purposes, conceptual bases, and 
developmental history of the CPS. They include (1) an under- 
standing of the usefulness to haiidicapped students of career 
planning in general and of the CPS, Microcomputer Version, in 
particular; (2) a description of the content, organization, and 
sequence of student activities of the CPS; (3) a detailed ex- 
planation of the procedures for using the management component 
of the CPS and for guiding students through the instructional 
component; (4) suggestions for group activities to supplement and 
expand upon the individualized core activities of the CPS; and 
(5) source!^ of additional information about careers for the 
instructor and student who wish to pursue various topics in more 
detail ♦ The Instructor Guide is thus the vehicle that integrates 
all the various aspects of the Career Planning System, Microcomputer 
Version, into a coherent, easily implemented, and thoroughly 
documented instructional package. 
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DESCRIPTION OF THE EVALUATION EFFORT 

The evaluation of the Career Planning System, Microcomputer 
Version, involved both formative and summative aspects. The 
formative aspect was initiated simultaneously with the materials 
adaptation phase of the project and continued through the field 
test phase. The summative aspect ran concurrently with the field 
test of the adapted materials (September - November 1982), Each 
phase, along with each instrument used in the formal data collec- 
tion effort, is described next. (A copy of the instruments 
themselves are included as the appendix to this report.) 

Formative Evaluation 

Development procedures as well as standards of quality for 
both tl-3 computerized and the print-based CPS materials were set 
out in the Adaptation Plan , a document prepared early in the 
project and reviewed for appropriateness by a member of the 
faculty for exceptional children of The Ohio State University. 
Throughout the development phase, project staff conducted 
periodic reviews of the materials to ensure their conformity with 
the specifications delineated in this Plan. 

Several kinds of formative efforts were conducted during the 
field test -^hase of the project. At the National Center, project 
staff reviewed all the computerized and printed materials for 
instructional and technical quality and corrected all the errors, 
omissions, and so forth that were found. Staff also made regular, 
biweekly telephone calls to each field test teacher to gain 
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information about effective and ineffective aspects of the CPS 
as they v;ere discovered. At the conclusion of the field test 
effort, staff made site visits to two schools to discuss the 
experience with various school personnel. These conversations 
provided National Center staff with the opportunity to probe 
into and compile extensive notes on the participating teachers • 
perceptions of the CPS. 

At each of the five field test sites, participating teachers 
and students also provided information throughout the testing 
effort in order to guide the revision of the CPS. Teachers used 
an Instructor Log bound into the "Evaluating the CPS" section of 
the Instructor Guide to provide comments on the strengths and 
weaknesses of the package and to offer suggestions for improve- 
ment. Teachers also completed Exit Interview Forms with students 
as they completed Interest Areas in order to glean the students' 
reactions to the Interest Areas and to form and record their own 
opinions of the succes^fe of the Interest Areas in imparting career 
knowledge and self -awareness to the students. 

Summat^ve Evaluation 
The primary aspect of the summative evaluation 3ffort in- 
volved a classical experimental design employed to test whether 
students in the treatment group did, in fact, gain more knowledge 
about themselves, about careers, and about the process of career 
planning than did their counterparts in the control group. Figure 
5 displays the general framework for this aspect of the summative 
effort . 
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LOCATION 
SAMPLE 
PRETEST . 



Experiinental Group '^ 

2 

All 5 test sites 
3 

49 students 
Categories: A, C 



INTERVENTION Microcomputer Version 

of CPS for ten weeks 



1 * 

Control Group 

2 

All 5 test sites 
2 3 students^ 
Categories: A, C 
None 



POS^TEST 



Categories: A, C, D Categories: A, B, C 



CATEGORIES: A — Measures of knowledge about careers 

B — Measures of knowledge about career planning 
in general 

_ ' C — Measures of knowledge about self with respect 
to careers and career planning 

s 

D — Brief questionnaire designed to obtain 
students* reactions to the CPS experience 



Students were randomly assigned to each group at each 
test site. 

'Boulder (Denver), Colorado; Brentwood (St. Louis) , 
Missouri; Carrollton (Dallas) , Texas; Columbus, Ohio; 
and Freeport (New York City) , New York* 

^Students out of the. orginally selected 50 (10 at each 
site) for whom complete data Were returned. 

^Students out o£ the originally selected 2 5 (5 at each 
site) for whom complete data were returned* 



Figure 5. 



Overview of the Design Used for the 
Formative Evaluation 
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Thi^ design was a classical experimental one inasmuch as the 
two groups were formed through random assignment, so that two 

t. 

^ groups at each site coald be considered comparable before the 
experiment began! During the course of the experiment^ the 
experiences of the two groups were identical, with the single 
exception of the intervention of the CPS^ Microcomputer Version^ 
in the activities of .he experimental group. The scores of the 
experimental group provided a posttreatment status indicator, 
whereas th scores of the control group provided a point of com- 
parison with the achievement of the experimental group students. 
The data from each csf the five test sites were analyzed sepa- 
rately and in combination with data from the other four sites to 
provide a variety of measures of statistical significance. 

The pretest, which was identical for both groups, contained 
thirteen questions (see the appendix) designed to obtain measures 
of each student *s perceptions of the three types of knowledge 
indicated in figure 5, which"ar'e "as "follows : 

• Category A — Knowledge about careers 

(questions 1, 7, 12) 

• Category B — Knowledge about career planning 

(questions 8, 9, 10, 13) ; 

• Category C — Knowledge about self with respect 

to careers and career planning 
(questions 2, 3, 4, 5, 6, 11) 

The posttest for both groups contained the identical thir- 
teen questions contained on the pretest, plus (in the case of 
the experimental group) ten additional questions designed to 
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obtain these students' reactions to the CPS, Microcomputer 
Version, These lattfer questions , which constitute category D 
of figure 5, are not considered to be part of the summative 
evaluation but were included to provide additional formative data. 

Responses on the pretests and posttests were coded at the 
partially ordered level of measurement (Coombs 195 3) as follows: 

1. Questions 1^ 2^ 3, 4^ and 5 were coded — 

a) Nothing at all = the real number 1 

b) .Very little = the real number 2 
^ c) I can ' t say - the real number 3 

d) Some = the real number 4 

e) A lot the real number 5 

2. Questions 1, 8, 10, 11, and 12 were coded — 

a) Agree = the real number 1 

b) Disagree = the real number 3 

c) I can't say = the real number 2 

3. Questions 6, 9, and 13 were coded-- 

a) Agree = the real number 3 

b) Disagree = the real number 1 

c) I can ' t say = the real number 2 

The assignment of real numbers in this manner made the 
''better" student response correspond to the higher numbers. As 
noted in Abelson and Tukey (1970 ) and in Labovitz (1970), the 
assignment of numeric values to the 6ategories in such a partially 
ordered level of measureinent allows the use of the properties of 
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the" real-number system in summarizing and analyzing measurements. 
As described in the. next section of this report, the Wilcoxon 
Matched-Pairs Sign Test' was used as a 'Statistical technique for 
analyzing the coded data in order to determine the effectiveness 

* f 

of the CPS, Microcomputer Version/ , . ' 

Additional summative data were obtained through the rating 
section of the Instructor Log* On this instrument, teachers were 
asked to rate ei'ght aspects of each instructional component of 
the CPS on "a five-point scale similar to a Likert scale* . They 
were also asked to use a similar scale to ra^e each section of 
the Instructor Guide according to its clarity, utility, and overall 
quality. Figure 6 displays both the items that were rated and the 
scale the teachers applied to express their ratings. Mean ratings 
were calculated in order to derive measures of teachers' opinions 
of the quality of the Career Planning System, Microcomputer Version. 



ITEMS RATED 


RATING SCALE 


Conceptual level 


5 = 


Very good, excellent 


Reading level 


4 = 


Good, more than acceptable^ 


Maturity level 


3 - 


Average , acceptable 


Level of diff s.ulty 


2 = 


Poor, minimally acceptable 


Motivational appeal 


1 ^ 


Very Poor, unacceptable 


Level of active student 
involvement 






Clarity 






Overall quality 







Figure 6. Teachers' Ratings of CPS Components 



ANALYSIS OF THE FORMATIVE DATA . 

As indicated in the previous section r several strategies 
were used to gather formative data, about the impact of the 
Caree' Planning System^ Microoomputer Version^ upon the students 
who participated in its field test. Students typically spent an 
average of one hour per week interacting with the microcomputer 
during the ten weeks of the pilot-testing process. Students' 
peifceptions of their experiences with the package were garnered 
through. Exit Interview Forms, while teachers' perceptions of the 

4 

quality and utility of the materials were gained through Instructor 
Logs, biweekly telephone conversations, and site visits. The 
results obtained using these strategies — particularly by the 
data returned to the National Center on 182 Exit Interview 
(Reaction) Forms and seven Instructor Logs- ,. described and 
discussed below. 

The Instructor Log included in the Instructor Guide (see 
the appendix) provided the most efficient and effective vehicle 
for organizing all the formative data into a thorough and logical 
discussion of the strengths and weaknesses of the CPS, Micro- 
computer Version. The Instructor Log was divided into sixteen 
sections that parallel the fifteen major instructional components 
of the package (Introduction, Interest Sort; each of t'he ten 
Interest Areas, Education Plan, Student Guide , and Instructor 
Guide ) and included a section fo-r ad-difeienal comments, as well. 
The Log was designed to encourage summary comments that would 
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identify the major strengths and weaknesses of the package and 
point the way to the most effective revisions* Extensive re- 
visions of the entire package were made by project staff, based 
on the suggestions made by instructors and students in the forma- 
tive evaluation process* The remainder of this section j.ntegrates 
all the relevant formative comments into the framework provided 
by the Instructor Log. What follows provided a framework for 
revision of the total system, 

CPS (Microcomputer) Introduction 

According to comments received from all the field test sites^ 
the CPS (Microcomputer) Introduction appears to be (before re- 
vision) one of the weakest parts of the CPS package. In general, 
although some students found this segment motivating, most seemed 
to find it too lengthy and/or too slow moving. Whereas the 
teachers . f el :•. the instructions were generally clear and the 
reading level allowed most students to work with complete in- 
dependence, the segment required too much reading and offered 
too few opportunities for active participation. 

Recognizing the need for students to acquire the information 
contained in the Introduction, the teachers offered several sug- 
gestions for imparting that information in more effective ways. 
These suggestions included (1) shortening the time loop in the 
early displays before the student takes 'jontrol of the program's 
pace; (2) replacing such vocabulary terms as "Probe" and "Brief" 
with simpler words; (3) eliminating some of the repetition provided 
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both on the computer and in the Student Guide activities designed 
to reinforce student learning; (4) allowing the students to 
choose among activities rather than moving linearly through the 
entire program; (5) reducing the amount of detail; and — as 
suggested by several teachers — (6) using more sound and graphics. 
Several teachers commented that the students enjoyed the word- 
search and crossword puzzles and suggested that more activities 
of this type would greatly imprCv/e this segment. 

I nterest Sort 

Teachers generally reported this segment to be clear, 
concise, and written at a level that allowed students to work 
with almost total independence. One teacher reported that the 
students were very interested in this segment and that the 
questions encouraged them to think about their likes and dis- 
likes, whereas another teacher felt that the students actually,., 
gave little thought to their answers. Several teachers raised 
the issue of whether the questions do, in fact, identify par- 
ticular Interest Areas, since many of the students identified as 
many as eight or nine Interest Areas. Because the questions 
were, in fact, designed to expand students' awareness of pos- 
sible interests rather than restrict their exploration to only 
a few^ several teachers felt it infeasible (especially in light 
of the time restrictions imposed by the field test deadline) to 
have students work on more than two or ^ihree Interest Areas in 
all. 
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Interest Areas 

During the field test^ students as well as teachers pro-- 
vided data on this core segment of the CPS^ Microcomputer Version 
Since exploration in each Interest Area — unlike movement through 
the Introduction and Interest Sort — was selective rather than 
universal^ data reported, in relation to each Interest Area were 
based on a narrower set of experiences than the data reported 
previously. ^These data^ as well as the number of students and 
field test, sites that provided data for each Interest Area, are 
reported in the discussions that follow. 

Advising (Sixteen Students at Four Sites) 

Students seemed generally to like these Activities and to 
feel they were neither too easy nor too difficult. Students 
reported a variety of specific likes (e.g., working with children 
figuring amounts, meeting people) and dislikes (e.g., low salary, 
sitting behind a desk, getting up early to go to work) about 
occupations presented in the Area. They seemed most attracted 
'to the ccupation of child care attendant and least interested 
in that of employment counselor. The teachers, too, seemed to 
like the child care attendant Activity best. Several teachers 
questioned the appropriateness of the lawyer occupation for. this 
group of students and noted that the amount of reading this 
Activity . required presents some difficulty. Overall, the 
teachers felt that nine of the sixteen students, acquired reason- 
able amounts of career knowledge and self-awareness from working 
in this Interest Area, while the other seven students did not, 
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Arrang in g (T w enty- two Students at l^'iv o Sites) 

Students' comments reflect this range of interest in 
Arranging, some calling the Activities boring and "no far" and 
others expressing delight in working with their hands and seeing 
how things looked in the different Activities. 

.Interior designer was the occupation most favored by the 
students/ although each of the other three occupations in this 
Area was considered the favorite by at least two students. 
Teachers found the Interest Areas simple and easy to understand 
and repeatedly cited the graphics in the florist Activity as 
attractive and motivating. Teachers expressed a concern about 
whether architect is a realistic occupation for their students 
and felt the students gained more career knowledge than self- 
awareness from the Interest Area, because many of" the students 
are unable to think abstractly and to relate occupacional 
information to themselves. 

Building and Making (Thirty-eight Students at Five Sites) 

This was the most popular Interest Area* during the field 
test, attracting the greatest number of students. Most indicated 
that they liked learning about materials, building* things, and 
working with their hands and with tools. Four students said 
what they liked best about the Interest Area was meeting the 
workers and learning v;hat they do — an indication, it would seem, 
that students relate well to the CPS, Microcomputer Version, 
strategy of using cartoon "workers" to convey occupational 
information uo students in a personal way. 
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students' ability to identify with the workers is indicated 
by several other comments as well. One student indicated that 
what she liked least about the Interest Area was "being up so 
high "--a concern expressed by the imaginary roof er--whereas 
another student cited the danger of hitting herself with a. 
hamiTier-- --another of the roofer's concerns* Several girls noted 
that jobs in this Interest Area involve heavy work and concluded 
that this Area is only "for men." Four students mentioned some 
aspect of reading as what they liked least about the Interest 
Area^ while six others gave this designation to math. 

Comments on five Instructor Logs as well as in Vc 'Us 
telephone conversations suggest that the computation l ^ lired 
in this Interest Area (e.g.^ figuring the areas of surfaces, the 
amounts of materials required, and so forth) is excessive and 
beyond the ability levels of the students. Indeed, almost all 
the teachers' comments on this Interest Area centered on the 
difficulty and "passivity" of the math exercises, although 
several teachers wrote that the choice of occupations in the Area 
is well suited to their students' interests and abilities. 
Teachers were also generally positive in their assessments of 
the amounts of career knowledge and self-awareness students 
gained from their work in the Area, noting that twenty-two student 
(out of the thirty--eight for whom responses were given) gained a 
reasonable amount of career knowledge, and nineteen acquired a 
reasonable amount of self-awareness through their participation 
in the "Building and Making" Activities and Briefs. 
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Doing Clerical W ork (Twent:y-twc Students at Five Sites) 

Students liked working with money and a "cash register," 

giving change, t^yping, and doaling with and helping people. One 

student liked physical work best^ whereas another liked lifting 

least. Two students indicated that stacking stock items in 

their bins was too easy, and one concluded that he would not 

like working alone and indoors in this job. 

Teachers' comments reflect the middle-of-the-road reception 

this Interest Area seemed to get from the students. Two 

Instructor Logs, for example, contained no comments at all, 

whereas one stated only that the one student who had tried the 

Area had enjoyed doing the Activities. Other teacher comments 

suggested that some of the Activities, although they seemed to 

interest the students, were perhaps too long or not active 

enough. The teachers generally thought the selection of occu- , 

pations in this Area is good but suggested that only about half 

of the students who selected the Area gained reasonable amounts 

of career knowledge and self-awareness from it. 

H elping (Nineteen Students at Five Sites) 

Students consistently expressed a strong interest in helping 

# 

people as their reason for enjoying these Activities and Briefs.' 
Police officer — the most popular occupat ion--was mentioned by 
eight students as an Activity they particularly enjoyed. Even 
though str ts' comments about what they liked least (i.e., the 
risk of shooting someone or being shot) indicate an awareness of 
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the dangers of this occupation^ this awareness seemed not to 
diminish interest in it. ^ 

Teachers* reactions to the Area ranged from one comment 
saying that overall it is quite good, to two comments suggesting 
it is rather weak. Teachers singled out the licensed practical 
nurse Activity as being excellent and sspecially realistic but 
felt the properties manager was not as well accepted, because 

this occupation and the theatrical world it represents are beyond 

•J 

the experiential and maturity levels of the studentfs. Teachers 
felt that more than half of their students gained reasonable 
amounts of career knowledge and self-awareness through the 
Interest Area. Teachers* negative responses to questions about 
these issues focused (as they did throughout) on problems of 
individual students* immaturity, low ability levels, failure 
to take career planning seriously, and so forth, rather than in 
terms of deficiencies in the CPS materials. 
Maintaining and Repairing (Sixteen Students at Five Sites) 

Student comments indicate that the auto mechanic Activity 
was by far the most popular choice, although the remarks seem 
to reflect previous interest in and familiarity with this occu-- 
pation rather than any particular appeal of the CPS presentation 
of it. 

Teachers at three sites indicated that the Interest Area 
was done very well and that it presented a set of. occupations 
to which their male students could easily relate. Staff at a 

2 6 

35 

ERIC 



fourth site called for a better. selection of occupations and 
activities (in particular^ the elimination of the sanitation 
worker Activity and the replacement of the auto machanic's 
form-completion activity with something similar to the "trouble- 
shooting" activity Qf the appliance repairer). Staff at the 
remaining site indicated that the reading was difficult for the 
students^ but praised the form-completion activity because it 
draws on retrieval, a skill that students can master. All 
teachers seemed to feel that only half of their students gviined 
reasonable amounts of career knowledge and self-awareness through 
this Interest Area. 

T hinking in Pictures (Fourteen Students at Five Sites) 

\ Students mentioned photographer and cartoonist as their 
favori^te Activities and named drawing, selecting subjects, and 
using the joystick as their favorite tasks. Teachers, too, 
commented on their students^ enjoyment of and facility with 
joysticks and suggested incorporating more joystick-based 
Acti^'ities into the CPS, Microcomputer Version, package. 

One teacher was particularly positive about this Interest 
Area, pointing out that it is enjoyable, very visual, filled 
with active involvement, and successful in giving students 
insights into jobs they had never considered before. Another 
teacher questioned the inclusion of the Area at all and suggested 
that it does not give a realistic view of what jobs in this field 
involve. On the v/hDle, the teachers felt that their students 
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gained reasonable amounts of career knowledge (ten positive 
responses and one negative) and sel f-awareness ( eight positive 
responses and three negative) from working here. 
U sing Environmental Information (Ten Students at Four Sites) 

Students' comments highlighted the fun of using the joy-" 
stick and the fact that so much new information was included 
here^ and cited ^reading the stories and asking the questions 
(both factors^ presumably^ in relation to the Briefs) as the 
least-liked aspects. 

Teachers^ too, commented once again on the value of the 
joystick and the appeal of such visually based activities as 
that of the landscape gardener ♦ One teacher commented that 
the students* inability to identify with the job roles intro- 
duced in this Area kept the students from exploring it fully. 
Indeed^ the Interest Area fared poorly in the teachers' ratings 
of the career knowledge (five positive responses and two 
negative) and self-awareness (three positive responses and 
four negative) it imparted. 

Working with Equipment (Sixteen Students at Five Sites) 

Most students who worked here mentioned cook and computer 
service technician as their favorite occupations within the 
Interest Area. Some students felt the Activities were too easy, 
whereas others complained that the Area involved too much reading. 
Several teachers cited the computer service technician Activity 
as being especially good and well received, whereas one teacher 
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found it far too compMcated. Staff at three sites mentioned 
that the cook Activity needs to be more active and creative. 

» 

Teachers felt the Area did only a fair job of helping students 
gain career knowledge and self-awareness. 

Working with Numbers and Symbols (Nine Students at Two Sites) 
This was the least popular among the CPS Interest Areas. 
Perhaps the small number of students who worked here can be 
explained in par.t by the position of its title,, the last (alpha- 
betically) among the areas. For this '-^ason "Working with 
Numbers and Symbols" would automaticalxy be presented by the ' 
computer as the last among all the areas rated equally by any 
student. Explained differently, even though a student's 
responses on the Interest Sort may indicate this as one of five 
or six Areas of an equally high rank, it would be the last Area 
with that rank that the student would explore. Sinpe the field' 
test concluded before all the students could work in all their 
identified Interest Areas, it is possible to assume that a 
number of students who would have liked to work in this Area 
simply never got to it. It is also possible to assume that many 
students in the target population have little, interest in working 
with numbers and symbols. 

Several students noted that they best liked the Activities 
that were like working puzzles and that offered a lot to figure 
out. Two students cited teller as their favorite Activity, one 
student found the market research coder Activity too easy, and 
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another student mentioned that the interest Area involves too 
much reading. Teachers also found the teller Activity to be 
effective and to offer. both positive and negative appraisals 
of the market research coder and laboratory assistant Activitie 
Overall, teachers felt the Interest Area was effective in im- 
parting career knowledge and somewhat less so in encouraging 
self-awareness. 

Education Plan 
The Education Plan, which is the last major part of the 
instructional package of the Career Planning System, Micro- 
computer Version, was actually used by very few students. 
Staff at only one site provided comments to any extent, and 
these relate more to the corresponding section of the Student 
Guide than to the computerized material. Essentially, these 
comments offer suggertions for (1) providing more specific 
career preparation information through the Briefs (rather than 
providing it through the teacher-student conferences that are 
an integral part of the CPS) and (2) including a separate disk 
listing the education and training requirements for the various 
CPS occupations. 

Student Guide 

Although one of the teachers mentioned that this component 
of the CPS, Microcomputer Version, package was very well done 
and several teachers commented about its specific strengths, 
the teachers found several facets of the product that call for 
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improvement. Four teachers, for example, questioned the in- 
clusion of the Reaction Forms in the Guide . Two teachers noted 
that the students do not have the verbal skills to make effec- 
tive use of the forms. One teacher thought the questions should 
be more specific. Another found the questions so broad and 
repetitive that they encouraged the students to respond super- 
ficially in order to lessen the amount of writing they had to do* 
Staff at one site suggested putting all the Guide vocabulary 
(i*e.^ the "Work Words" and "Education Words") on the computer 
and eliminating the pages designed to reinforce student under-- 
standing of the goals and organization of the CPS package. 

Instructor Guide 
Staff at four of the sites found this Guide good or excel- 
Lent, but staff at one site found the instructions too wordy and 
suggested summarizing each section in outline form. These' 
teachers also suggested including a list of Interest Areas, With 
their corresponding disk numbers, in the Guid e. 

A dditional Comments 
This final section of the Instructor Log was designed to 
give participating teachers an opportunity to summarize their 
reactions to their experience with the CPS, Microcomputer Version 
and to offer any further suggestions for revision they felt war- 
ranted. This evaluation report integrates the teachers' general 
comments gleaned from the Logs as well as from the other data 
collection strategies described elsewhere in this report* 
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In general, the teachers felt that the instructional package 
embodies a good approach to the concepts of career planning, 
makes use of a delivery system that is intrinsically attractive 
to students in the target population, and is capable of engen- 
dering among thiS group the growth (to some degree) in career 
knowledge and self-awareness that is the package's primary goal. 
Teachers also pointed out some problems, however, some of which 
are inherent in the participating students and some of which can 
be traced to the package itself. 

Teachers at several sites commented on their exceptional 
students' relative immaturity and suggested that the CPS mate- 
rials may be better suited either to older,' learning-handicapped 
students or to nonhandicapped persons of ail ages-. The field 
test students had difficulty in understanding the information . 
in the Introduction, in engaging in the introspective and 
future-directed thought necessary for career planning, and in ■ 
verbalizing the reactions on which their plans for the future 
would be made* Although they enjoyed and were able to complete 
the individual Activities in the package, they often seemed un- 
able to generalize beyond those discrete experiences to achieve 
a significantly greater amount of skill in career planning and 
decision making. • 

Several general aspects" of the CPS, Microcomputer Version, 
itself were seen as contributing negatively to the students^ 
experiences. Teachers commented repeatedly that the materials • 
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are too wordy, the reading level high, and the activities 

too involved. Teachers strongly suggested that the CPS should 
include more games and simulations, more visual stimuli, and 
more use of the joystick. Teachers also called for less reading, 
less question answering, and less math. 

Several very important coininents made by instructors during 
the site visits or during the biweekly telephone calls from proj- 
ect staff were not recorded on the Instructor Logs. Some were 
concerned with the necessity of using the joystick and keyboard 
simultaneously; others felt that the use of computer graphics 
(although effective) leaves much to be desired when compared to 
the quality of a television picture such as might be delivered 
through the use of a videodisc player. Some teachers favored a 
less structured management system. Many felt there was too much 
reading; some asked for a printer to record Interest Sort results. 
All teachers held the opinion that CPS, Microcomputer Version, 
would be a valuable tool for career planning preparation for a 
wide range of persons and ages. They suggested rhat the system 
not be limited to special education, middle school children. They 
encouraged pro'ject staff to disseminate the system widely. 

In summary, the teachers reported* that the students enjoyed 
working with the Career Planning System, Microcomputer Version, 
as well as working on the computer itself, and that some students 
seemed to gain significantly from the experience. Overall, however 
the teachers saw ""he need for a number of revisions before the pack 
age can be truly useful for most of the students for whom it was 
designed. 
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ANALYSIS OF THE SUMMATIVE DATA 

As explained in the section, "Description of the Evaluation 
Effort," summative data about the impact of the Career Planning 
System, Microcomputer Version, upon the field test participants 
were gathered through two evaluation instruments. Data regarding 
the effectiveness of the materials upon student growth were 
obtained through the administration of pretests and posttests to 
the exp 3rimental-group and control-group students. Data re- 
garding the teachers' assessments of the overall quality and 
utility of the CPS, Microcomputer Version, were elicited through 
a five-point rating scale (similar to a Likert scale) that was 
included on the Instructor Log. This section of the report 
presents and discusses the information obtained through the 
analysis of both kinds of data. 

D ata Indicating Student Growth \ 
The expected impact of this project was that, as a result 
of their intera'^tion with tho microcomputer version of CPS, 
students in the experimental groups would gain more knowledge 
(about careers, about self with respect to careers and career 
planning, and about career planning in general) than would 
students in the control groups. 

In order to test this hypothesis in a global fashion, the 
experimental groups at all five pilot test sites were aggregated 
to form one large experimental group (n - 48) . Similarly, the 



five control groups were aggrogated to form one large control 
group (n = 23). Next, for each of the aggregated groups cal- 
culations were made of the mean scores on each of the thirteen 
common questions contained in the pretest and the posttest. 
From these mean scores, calculations were made of the mean 
changes made by each group for each question. These results 
are shown in ■■. ible 1 for the aggregated experimental group and 
in table 2 for the aggregated group control group. Finally, 
table 3 displays a comparison of the changes between the pre- 
test s;,cores and the posttest scores made by each group for each 
question. 

To analyze these changes statistically, the use of a non- 
parametric or distribution-free statistical approach was neces- 
sary, since no assumptions could be made about the sample dis- 
tributicns of either the aggregate experimental group or the 
aggregate control group. The statistical approach selected was 
the Wilcoxon Matched-Pairs Sign Test (Downie and Heath, 1965). . 
Th^ chief advantage of this application is its avoidance of all 
violations of parametric assumptions when sample sizes' are less 
than or equal to ten. The experimental design selected at the 
start of the project called for no sample sizes greater than ten. 
Despite random selection of subjects, such small samples (at 
each pilot-test site) made it unwise to assume a parametric 
distributiion of any type. 
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TABLE 1 

AGGREGATED EXPEUiMENTAL GROUP MEAN SCORES (n=48) 

Mean Mean 

Question Pretest Score Posttest Score Mean Change 

1 3.292 3. 87.5 +0.583 

2 3. 500 3. 938 +0. 438 

3 3.771 4.021 +Q.250 

4 3.771 4.125 +0.354 

5 3.500 4.000 +0.500 

6 2.854 2.813 -0.041 

7 2.167 2.146 -0.021 

8 2.042 2.208 +0.166 

9 2.833 2.708 -0.125 

10 2.396 2.458 +0.062 

11 1.938 ' 2.063 +0.125 

12 2.063 2.292 +0.229 

13 2, 729 2. 85,4 +0. 125 
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TABLE 2 



AGGREGATED CONTROL GROUP MEAN SCORES (n=2 3) 



Question 



5 



IC 



11 



12 



13 



Mean 
Pretest Score 

2.652 



3. 652 
3.913 
3.478 
3.304 
2.913 
2. 5 32 
2.478 
2. 870 
2. 522 
1.957 
1.783 
2.652 



Mean 
Posttest Score 

3. 34 8 



3.087 



4.043 



4.000 



3.913 



2.739 



2. 348 

V 

2. 304 



2.652 
2. 391 
1.913 
2.000 
2.696 



Mean Change 
+0.696 

-0.565 

+0. 130 

+0. 52 0 

+0. 609 

-0. 174 

-0. 174 

-0. 174 

-0.218 

-0. 131 

-0.044 

+0. 217 

+0. 044 



ERIC 



38 



46 



TABLE 3 



COMPARISON or CHANGES MADE BY EACH GROUP 



(1) 



Question 



2 
3 



5 
6 
7 
8 
9 
10 
11 
12 
13 



(2) 

Change in 
Aggregated 
Experimental 
Group (n=48) 

+0. 583 



+0.4.^8 
+0.250 
+0. 354 
+0. 500 
-0.041 
-0. 021 
+0. 166 
-0. 125 
+0. 062 
+0. 125 
+0.229 
+0. 125 



(3) 

Change in 
Aggregated 

Control 
Group (n=2 3) 

+0. 696 



-0.565 
+0. 130 
+0,520 
+0. 609 
-0. 174 
-0. 174 
-0. 174 
-0. 218 
-0. 131 
-0. 044 
+0.217 
+0. 044 



(2-3) 



Difference 
in Changes 

-0. 113 



+1.003 
+0. 120 
-0. 166 
-0. 109 
+0. 133 
+0. 153 
' +0. 34 0 
+0.093 
+0. 19 3 
+0. 169 
+0.012 
+0.081 
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T esting the Hypothesis 
The null hypothesis (that there is no difference in the 
changes made by the aggregated experimental group and by the 
aggregated control group) is stated in terms of the number of 
signs (positive or negative) in. the last column of table 3. The 
null hypothesis tested is that the median change is zero. This 
hypothesis is rejected if there are too few plus signs (one- 
tailed test) . The test of significance used is the binomial 
distribution. 

For the binomial calculation, n=l3 (the number of questions) 
and p=0.5 (the hypothesis that both groups are still from the 
same population despite the pro ject intervention) . In table 3 
ten pluses and three minuses are noted. By chance there could 
be six or seven of each expected. The question is, does this 
frequency of ten pluses differ significantly from what would be 
expected by chance? A table of binomial probabilities for n=13, 
p=0 . 5 yields the following: 

.000122 + .001586 + .009516 + .034892 = .046116 
Thus the probability of obtaining, ten or more plus signs is 
0.046116, or less than 5 percent. This enables the null hypothesis 
to be rejected at the .05 level. 

The conclusion is that the two groups are not from the 
identical population following project intervention. It can 
therefore be stated that, as a result of their intervention with 
the microcomputer version of CPS, students in the aggregated 
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c^xpGrimental group qainod more knowiodqe about careers, about 
self 'with respect to careers and career planning, and about 
career planning in. general, than did students in the aggregated 
control group This difference is significant at the .05 level. 
A one-tailed test was used because the hypothesis stated that a 
greater gain- would be made by the experimental groups. 

Implications of the Summative Results 
Significance at the .05 level on a one-tailed binomial test 
is not an especially strong result. It should be noted, however, 
that the pilot-^testing procedures employed were designed to 
simultaneously accomplish four needs: (1) to match the structure 
of a typical middle school education classroom situation; (2) to 
utilize, instruments employed during the field testing of the 
earlier print version of the CPS; (3) to obtain suggestions for 
revision through the collection of formative data; and (4) to 
obtain summative data. Therefore, some compromises were neces- 
sarily made: (1) a maximum class size of fifteen special educa- 
tion students limited the experimental group to ten students and 
the control group to five students at each site; (2) only ten 
weeks were allowed for pilot-testing so as not to unduly inter- 
fere with the planned curriculum at each school; (3) new instru- 
ments were not designed because the project was funded as an 
adaptation of an earlier product that included instruments; and 
(4) in order to obtain a wide variety of formative data, no 
requirement was made to have homogeneity among the test 
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sites — thus excluding any basis for comparing summative data 
gathered from different sites. 

The compromises listed previously do not detract, however, 
from the overall conclusion that the summative data from the 
aggregated groups supports the -conclusion that as a result of 
their interaction with the microcomputer version of CPS, students 
in the experimental groups gained more knowledge (about careers, 
about self with respect to careers and career planning, and 
about career planning in general) than, did students in the control 
groups . 

Analysis of Teachers' Ratings 
In order to provide their summary opinions of the CPS, 
Microcomputer Version, participating teachers assigned numerical 
ratings to each computerized component of the package, to its 
Student Guide , and to its Instructor Guide . Teachers based their 
ratings on a five-point scale, similar to a Likert scale,* which 
they applied as ap jpriate to each item under consideration. 
All the student materials — that is, each of the thirteen com- 
puterized CPS components and the print-based Student Guide — were 
rated according to their perceived appropriateness for the students 
along the eight dimensions displayed in table 4. Each of the four 

^According to~Ehis scale, 5 = very good, ex»^ellent; 4 = good, 
more than acceptable; 3 = average, acceptable; 2 - poor, 
minimally acceptable; and 1 ~ very poor, unacceptable. 
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sections of the Instructor' Guide — the. introduction , technical 
procedures, instructional procedures, and suggestions for sup- 
plementing the package — was rated along the three dimensions 
displayed in table 5. 

An inspection of table 4 revoc,\s that the teachers perceived 
all the student materials to be generally in the acceptable-more 
than acceptable range and the computerized materials to meet 
students' needs more MOsely than the pi^int-based Student Guide , 
The level of active student involvement Was the most highly rated 
aspect of the computerized materials, while the level of diffi- 
culty received the lowest rating. ' Clarity and conceptual level 
were perceived as the strongest aspects of the Student Guide 9 
while motivational appeal was perceived as its weakest aspect 
and, indeed, as the only aspect close to the minimally acceptable 
range. 

Table 5 indicates that the Instructor Guide fared somewhat 
better than the student materials, achieving Ratings on all 
three dimensions in the good - very good range. Indeed, the mean 
ratings presented in this table may be somewhat misleading, since 
the modal rating for the Instructor Guide — reported in fifty-two 
of the total sixty-nine ratings across all sections an<S all 
dimensions — was "5 - very good, excellent.*' ^ 
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TABLE 4 




MEAN RATINGS 


OF STUDENT MATERIALS 




■ 1 

4I1M 

Item Rated 


Mean Ratings 

Computerized 
Components 


Student 
Guide 


Conceptual level 


3.62 


3,50 


Reading level 


3.62 


3. 33 


Maturity level 


3. 59 . 


3. 33 


Level of difficulty 


3.47 


3, 17 


Motivational appeal 


3.63 


2. 83 


Level of active student 
involvement 


3.66 


3.33 


Clarity 


3.57 


3.50 


Overall quality 


3.56 


3.00 



r 




TABLE 5 






MEAN 


RATINGS :pF INSTRUCTOR GUIDE 




Item 


Rated 




Mean Rating 


Clarity 






4.26 










Utility 






4. 30 


Overall 


quality 




4. 30 
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CONCLUSIONS AND RECOMMENDATIONS 

The analyses of the formative and summative data discussed 
in the two preceding sections have led to a number of conclu- 
sions about the field test of the Career Planning System, 
Microcomputer Version, as well as recommendations 'for the 
revision of its component materials. The -purpose of this section 
is to present those conclusions and recommendations that may 
assist researchers and developers undertaking future projects 
of a similar nature. 

fa 

Conclusions 
The main conclusions are as follows: 

# Middle- school students from a variety of special 
populations (mildly mentally retarded, learning 
disabled, and behavioral ly handicapped) found the 
system interesting, enjoyable, and usable. 

# Teachers of such students found the system bene- 
ficial for teaching career planning and for 
encouraging and facilitating career guidance 
interviews. 

# Students in the aggregated experimental group 
gained more knowledge (about careers, about self 
with respect to careers and career planning, and 
about career planning in general) than did students 
in the aggregated control group. The difference 
was statistically significant at the .05 level. 
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\ 

• The sysi^em .is usable for a wider audience than 

\ 

the groii^ with which it was pilot-tested. 

• The system's various programs^ if made accessible 

by means a simple menu device,, are valuable 

\ 

even without the management system. 

\ 
\ 

\ Recommendations 

\ n 

The main recoini;iiendations are as follows: 

• The CPS, Microcomputer Version, should be widely ' 
disseminated and used abong different populations 
(special education students and others) at a wide 
range of grade levels. 

• An alternative management system for CPS, Micro- 
computer Version, , should eventually be prepared 
to allow access to any program through a simple 
menu device, without requiring the user to first 
pass through the Interest Sorti 

• Print-based materials should not be adapted to 
the microcomputer; the inevitable tendency is to 
make the microcomputer an electronic page turner. 

• The use of a microcomputer with a videodisc 
player (or similar device) should be investigated 
to provide better sound and graphics while (pos- 
sibly) reducing the programming time. 

m When possible^ systems should not require the 

simultaneous use. of the joystick and the keyboard. 
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• When possible ; hard copy, such as the results 
of the Interest Sort, should be prepared by a 
printer attached to the microcomputer, rather 
than by students or teachers who make hand- 
written entries in an accompanying manual. 

• The -research design for evaluating such projects 
should involve larger pilot-test groups and a 
greater number of questions for each category' 
of information investigated. 

• The length of the pilot-test period for such 
projects should be one school year, rather than 
one quarter, in order to enable more students 
to complete the entire system. 
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CPS student Feedback Form 



NAML" 



SCHOOL 



TEAC!!FR 



Di rections : 

Read the question. Choose the answer that says how you feel. Check (v^ ) 
the space next to that answer. 

1. How much do you know about the work done in different jobs? 

(a) Nothing at all. 

(b) Very little. 

(c) I can't say. 

(d) Some. 

(e) A lot. • 

2. How much do you know about how your personal interests and abilities 
are related to the work that is done in different jobs? 

(a) Nothing at all. 

(b) Very little. 

(c) I can't say. 

(d) Some. 

(e) A lot. 

3. He . much do you know about which jobs you might like and which jobs you 
might not like as well? 

(a) Nothing at all. 

(b) Very little. 

(c) I can't say. 

(d) Some. 

(e) A lot. 

4. How much have you thought about your future? 

(a) Not at all , 

(b) Very little. 

(c) I can't say. 

_(d) Some. 

(e) A lot. 

5. How much have you thought about which studies and activities you should 
choose? 

(a) Not at all . 

(b) Very little. 

(c) I can't say. 

(d) Some. 

____ (e) A lot. 
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6. You have to know what you're good at and what you're p'or at before 
you can make good decisions about your career. 

(a) Agree, 

(b) Disagree, 

(c) I can't say. 

7. I know all that I need to about the job(s) that I'm interested in. 

(a) Agree. 

(b) Disagree. 

(c) I can't say. 

8. Your career is decided by chance, 

(a) Agree. 

(b) Disagree. 

(c) I can't say. 

9. You should think about what you are good at and_ what you like when 
making career decisions. 

(a) Agree. 

(b) Disagree. 

(c) I can't say. 

10. I won't worry about planning and preparing for my career; I know 
something will come along sooner or later. 

_(a) Agree. 

" ( b ) Disagree. 

(c) I can't say. . . 

11. So far I haven't been able to find an occupation that I would rpally 
like to get into. 

(a) Agree. 

(b) Disagree, 

(c) I can't say, 

12. I know very little about job requirements. 

(a) Agree. 

(b) Disagree, 

(c) I can't say, 

13. In planning for your career you need to know what kind of person you are, 

(a) Agree, 

(b) Disagree. 

(c) I can't say. 
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Instructor Log 



Please use the scale at the right 
to evaluate each component of the 
Career Planning System^ micro-- 
computer version, listed below. 
Be sure to add your comments 
(strengths, weaknesses, sugges- 
tions for improvement ) in the 
spaces provided* 



^ — 

4 = 

3 = 

2 = 

1 - 



very good, excellent 
good, more than acceptable 
average , acceptable 
poor, minimally acceptable 
very poor, unacceptable 



INTRODUCTION 

Conceptual leve 1 

Reading level 

Maturity level 

Level of difficulty 

Motivational appeal 

Level of active stu- 
dent involvement 

Clarity 

Overall quality 



INTEREST SORT 

Conceptual level 

Reading level 

Maturity level 

Level of difficulty 

Motivational appeal 

Level of active stu- 
dent involvement 

Clarity 

Overall quality 



Rating 



Coiranents 



n5 
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Rating 



Comments 



INTEREST AREAS 

1, Advising 

Conceptual level 

Reading level 

Maturity level 

Level of difficulty 

Motivational appeal 

Level of active stu- 
dent involvement 

Clarity 

Overall quality 



2, Arranging 

Conceptual level 
Reading level 
Maturity level 
Level of difficulty 
Motivational appeal 

Level of active stu- 
dent involvement 

Clarity 

Overall quality 
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Rating 



Comments 



INTEREST AREAS (cont'd.) 

3. Building and Making 

Conceptual level 

Reading level 

Maturity level 

Level of difficulty 

Motivational appeal 

Level of active stu- 
dent involvement 

Clarity 

Overall quality 



4. Doing Clerical Work 

Conceptual level 

Reading level 

Maturity level 

Level of difficulty 

Motivational appeal 

Level of active stu- 
dent involvement 

Clarity 

Overall quality 
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INTEREST AREAS (cont'd.) 

5. Helping 

Conceptual level 

Reading level 

Maturity level 

Level of difficulty 

Motivational appeal 

Level of active stu- 
dent involvement 

.Clarity 

Overall quality 



6. Maintaining and 
Repairing 

Conceptual level 

Reading level 

Maturity level 

Level of difficulty 

Motivational appeal 

Level of active stu- 
dent involvement 

Clarity 

Overall quality 



INTEREST AREAS (cont'd.) 

7. Thinking in Pictures 

Conceptual level 

Reading level 

Maturity level 

Level of difficulty 

Motivational appeal 

Level of active stu- 
dent involvement 

Clarity 

# 

Overall quality 



Rating 



Comments 



8. Using Environmental 
Information 

Conceptual level 

Reading level 

Maturity level 

Level of difficulty 

Motivational appeal 

Level of active stu 
dent involvement 

C lar ity 

Overall quality 
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Rating 



Comments 



INTEREST' AREAS ( cont ' d . ) 

9. Working with Equipment 

Conceptual level 

Reading level 

Maturity level 

Level of difficulty 

Motivational appeal 

Level of active stu- 
dent involvement 

Clarity 

Overall quality 



10. Working with NuiiOoers 
and Symbols 

Conceptual level 

Reading level 

Maturity level 

Level of difficulty 

4 

Motivational app*2al 

Level of active stu 
dent involvement 

Clarity 

Overall quality 
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EDUCATION PLAN 

Conceptual level 

Reading level 

Maturity level 

Level of difficulty 

Motivational appeal 

Level of active stu- 
dent involvement 

Clarity 

Overall quality 



Rating 



STUDENT GUIDE 

Conceptual level 

Reading level 

Maturity level 

Level of difficulty 

Motivational appeal 

Level of active stu- 
dent involvement 

Clarity 

Overall quality 



INSTRUCTOR GUIDE 

Introducing the CPS 
Clarity 
Utility 

Overall quality 



Comments 
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Rating 


Comments 


INSTRUCTOR GUIDE (cont'd.) 






Using the CPS 






A. Technical 
procedures 






Clarity 






Utility 






Overall quality 






B. Instructional 
procedures 






Clarity 


* 




Utility 






Overall quality 






Supplementing the CPS 






Clarity 






Utility 






Overall quality 











ADDITIONAL COMMENTS: 
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